Call Admission Control in Wireless Ds-cdma Systems Using Reinforcement Learning

THAI) สาขาวิชาวิศวกรรมโทรคมนาคม ลายมือช่ือนักศึกษา ปการศึกษา 2549 ลายมือช่ืออาจารยที่ปรึกษา PITIPONG CHANLOHA : CALL ADMISSION CONTROL IN WIRELESS DS-CDMA SYSTEMS USING REINFORCEMENT LEARNING. THESIS ADVISOR : ASST. PROF. WIPAWEE HATTAGAM, Ph.D. 95 PP. ABSTRACT (ENGLISH) DIRECT-SEQUENTIAL CODE DIVISION MULTIPLE ACCESS (DS-CDMA)/ CALL ADMISSION CONTROL/ REINFORCEMENT LEARNING/ ACTOR-CRITIC REINFORCEMENT LEARNING (ACSMDP)/ SEMI-MARKOV DECISION PROCESS (SMDP) The underlying aim of this research is to find the best possible call admission control policy for multiclass voice services in wireless direct-sequential code division multiple access (DS-CDMA) systems that maximize the long-term reward of the system while satisfying multiple quality-of-service (QoS) constraints. The call admission control problem is formulated as a semi-Markov decision process. Two important constraints inherent in CDMA systems are considered which are signal-to-interference ratio (SIR) constraints and blocking probability constraints. To circumvent the computational burden of conventional dynamic programming (DP) methods, we employ an actor-critic reinforcement learning method to solve the call admission control problem. Furthermore, we incorporate a penalty function into the reward signal in order to account for the blocking probability constraints. The SIR constraints are accounted for by embedding them into the system state space.